Pounder II is the second tool in a series of Web server testing applications from the nice people at BIAP Systems, Inc. Pounder II's predecessor has been one of the most underdocumented, difficult to obtain, but highly sought after pieces of Mac networking software for good reason. It was ugly, prone to destructive behavior, and a general embarassment to its author. Hopefully, Pounder II corrects most of the shortcomings in its predecessor.
So what does it do? Pounder II is a load generator for Web sites. Much like WebStone and Pounder, Pounder II can place an arbitrarily large or small load of simulated HTTP client connections on a specific Web server for test and evaluation purposes. Unlike WebStone and Pounder, Pounder II can be used in a completely repeatable fashion. That is, the test sequences that Pounder II generates are reproducible if you so desire.
New features in Pounder II include complete logging of responses from the Web servers being tested, the ability to test multiple servers at once (e.g., RAIC performance testing), real-time status reporting of client/server performance measures, a larger number of simulated clients, and greater control over the sequencing and timing of HTTP requests.
2.0 Installing and Configuring Pounder II
Simply place the Pounder II application and the "Pounder II.config" file in the same folder to complete the installation. You must modify the config file to reflect the host address(es) you wish to "pound", as well as customize any other settings that affect Pounder II behavior. Pounder II is a "fat" binary and should run on any Mac with MacTCP or Open Transport installed. It only uses the MacTCP interfaces, so it is limited somewhat in performance on OT machines and has an upper limit of 50 simulated clients.
Note that Pounder II will not operate if the server(s) targetted by Pounder II does not contain a file called "pounder.gif" in its top level directory. This prevents Pounder II from being used in denial of service attacks, etc. The file can be empty and doesn't have to be an actual GIF image. It just needs to be present and Pounder II needs to be able to receive a "200 OK" status response from the server(s) being tested when this URL is requested (http://your.host.com/pounder.gif).
3.0 Config File Format
The "Pounder II.config" file is the heart (and a fragile one at that) of the Pounder II application. It contains all the information necessary to configure and execute tests of multiple HTTP servers. The format of the config file is fairly structured. The included config file contains comments that describe each setting, its syntax, etc. Details on the format and settings are as follows:
Line 1: Version number - this version number must match the version of Pounder II that it is being run with.
Line 2: Log file name - this is the name of the Mac text file that will receive the output from e Pounder II application. Pounder II will create the file if it doesn't exist, and will overwrite it if it does. The file name shouldn't contain any spaces and should be a simple file name, no ":" characters. If this line is blank (or the first character is a space), Pounder II will not create or write to a log file. This is useful if you just want to generate a load on your server and don't want a hard disk full of statistics.
Line 3: Log file format - this line should consist of a single character , 0, 1, or 2. This specifies the format of the log file. 0 = summary reports for each connection only, 1 = summary reports in addition to complete HTTP event reporting, and 2 = event reporting only. See the section on Log Formats below for more details.
Line 4: Sequential or random requests - this line should consist of a single character, 0 or 1. 0 means that each HTTP request listed in the config file will be submitted to the server by a given "pounder" (HTTP client thread) in the order specified in the config file. If this line is 1, the requests will be submitted in a random order. Random ordering is not implemented in this version of Pounder II.
Line 5: Absolute or random delay - Each "pounder" in Pounder II will delay a specified amount after a HTTP request completes before submitting the next request. If this line is 0, the delay will be a constant time that is specified as part of each request definition. If a non-zero value is specified, it represents a random number of ticks between 0 and the number specified that the pounder will wait before submitting the next request. Ticks are 1/60th of a second.
Line 6: Repeat count - this line specifies the number of iterations each "pounder" will make through the list of HTTP requests specified in the config file. If this number is zero, Pounder II will repeat the tests until the user stops execution by typing command-period or quitting the application.
Line 7: Number of simulated clients - this value controls the number of "pounders" that will be created to simulate client activity. Values can range between 1 and 50.
Comments: at this point in the file, an arbitrary number of blank lines or comment lines beginning with a "#" character may appear. It may be useful to add comments that describe the following HTTP requests, for example.
HTTP Requests - The remainder of the config file is a series of one or more HTTP requests. Each HTTP request begins with a line that specifies host and delay information, is followed by the actual HTTP request data, and is terminated with a line containing a single "@" character. Note that the config file must end with a single "@" character.
The syntax for the first line of the request looks like:
<host name or address> <port number> <delay> <response size>
An example of the first line of a request definition might be:
your.host.com 80 30 1550
where "your.host.com" is the domain name (or just an IP address) of the server that this request is to be sent to. "80" is the TCP/IP port number that the destination server is listening on. "30" indicates that the pounder should wait 1/2 second after completion of the request before proceeding to the next request (30/60ths of a second). "1550" says that the expected amount of data that should be returned in response to the request is 1550 bytes. Future versions of Pounder II will verify that the correct number of bytes were received by comparing the actual number received from the server with this value.
The next portion of the request is the actual HTTP request that a client like Netscape would send to a HTTP server. It follows immediately after the first line of the request definition that specifies host information. Actual HTTP requests must have a carraige return and linefeed at the end of each line. Pounder II simplifies this for you by allowing you to use normal Mac text files with only carraige returns at the ends of lines. It will add an additional line feed character to each HTTP request line, so don't add them yourself!
Last of all, a single HTTP request definition is terminated with a line consisting of a single "@" character. If this is the last definition in the config file, do not enter any additional characters (returns included) after the final "@".
4.0 Running a Test
Once the config file is properly set up, simply double click the Pounder II application. The display is fairly simple. The scrolling message area reports the parameters that will be used when testing begins. The status area at the top will display connection monitoring info on the left side and a summary of statistics being collected on the right side.
Before doing anything else, Pounder II confirms that it is OK to run tests against a given server. If you specify more than one host in your config file's requests, Pounder II checks all of them for a "pounder.gif" file. Any host missing that file will not be tested by Pounder II.
To begin running a test, select the "Run" command from the Control menu. To stop the test, select the "Stop" command. After quitting the server, you can consult the log file for specifics on the tests that were run.
The Display
The connection monitoring area in the upper left portion of the display will consist of a series of small, colored squares with each square representing the current activity associated with a given "pounder." Each color represents a different operation being performed by that specific HTTP client thread. The About box in Pounder II provides a key to the colors, but more on that later.
The statistics summary has 3 columns. The first column (with entries like 2xx, 3xx, 4xx, etc.) represents the number of HTTP requests that resulted in the server returning a specific HTTP response code. For example, a successful HTTP request usually returns a response code of 200, so that would increment the 2xx counter. If you select "Refuse Connections" from inside WebSTAR, the server will begin returning 503 error codes (at least newer releases will), which will increment the 5xx counter. You will also notice that the colors associated with the connection monitors match up with status codes, green for 2xx series responses and red for all others. This gives you some idea of whether your HTTP requests are being processed as expected or not. Remember that often it is normal to get a non-2xx response.
The second and third columns are pretty self-explanatory. The measures in these columns only apply to the last (or current) test to be run and are not cumulative. "Terminated" connections are those that were dropped by the server prior to completion of the HTTP request. "Can't Connect" measures the number of requests that were sent to a server but never accepted (i.e., no response.)
Connection area colors are interpreted as follows:
light gray - connection is being opened
yellow - the HTTP request is being sent
dark green - the HTTP response is being read from the server and is a 2xx series response code.
light green - the 2xx series connection is being closed
dark gray - the connection is idle and/or the specified delay period is being observed
dark red - the HTTP response is being read, and the response code is not in the 2xx series
light red - the non-2xx series connection is being closed
Connection thread number "0" is represented by the upper left colored square, thread number "1" is below it, followed by 2 and 3. Thread 4 starts the top of the second column and so on.
5.0 Log File Format
The log file has a fixed format, with varying degrees of detail being logged for each HTTP request. As mentioned above, you control the level of detail present in the log file. Each entry in the log file is a line containing tab separated values terminated by a carraige return and has the following format:
The <connection id> is an incrementing number that is bumped for each new HTTP request that is transacted with a server. Each "pounder" thread handles one connection at a time and when it's done, it gets a new connection id for the next request. Each connection ID is unique and all lines with the same connection ID are part of the same HTTP request transaction.
The <sequence number> is more or less a line number that is useful for returning the data to its original order if you are manipulating the log information in a spreadsheet application.
<opcode> is a text string (no spaces) that describes the particular operation that is being logged. Details on specific opcodes follow later.
<time stamp> is the "tick" value at the time the event being logged occurred. The difference between any the time stamps of any 2 operations in a given transaction will give you the duration of the first (lesser time stamp) operation in 60ths of a second.
<data values> are one or more value that is specific to the particular <opcode>. A summary of opcodes and their data values follows:
Opcodes and data values
The following opcodes are emitted to the log file if event logging is enabled (log format 1 or 2):
OPEN - marks the beginning of the attempt to contact the server specified in the request. Data argument is the ID of the thread (pounder) handling the request (normally 0 through 49).
OPEN_ERR - if the open command failed, marks the time the error happened. Data argument is the internal Mac O/S error code.
OPEN_FIN - marks the time the open request completed, indicating that the HTTP connection has been established. The data argument represents which HTTP request is being submitted in subsequent WRITE operations. (i.e., 3 means the 3rd HTTP request in the config file).
WRITE - start of transmission of HTTP request to server. No data.
WRITE_ERR - a TCP write error occurred, with data being the error code.
WRITE_FIN - marks the time that the HTTP request transmission was complete. No data.
READ - marks the point at which the pounder began reading the HTTP response from the server. No data.
STATUS - marks the time the first buffer of the HTTP response was received. Data is the HTTP response code (e.g., 200 OK)
READ_ERR - data is the TCP error code. Note that a -23005 error is normal and means there is no more data to read.
READ_FIN - time that the complete HTTP response was received. Data is the number of bytes contained in the complete response.
CLOSE - time that the connection close was issued by the pounder.
CLOSE_ERR - data is the error code if the close fails.
CLOSE_FIN - marks the point that the complete HTTP request and response is complete. No data.
The following opcode is emitted if the log format includes summaries (0 or 1):
SUMMARY - written at the end of a complete HTTP request and response. Data consists of 7 integer fields: <OPEN tick> <WRITE_FIN tick> <READ tick> <READ_FIN tick> <CLOSE_FIN tick> <HTTP status code> <bytes sent>
The "tick" values are the time stamps from the opcodes of the same name, with the last 2 fields being the HTTP response code and the number of bytes sent by the server to the pounder.
6.0 Interpreting the Results
As with any statistics generating tool, the potential for abuse exists with data produced by Pounder II. First and foremost, the stats displayed by a single copy of Pounder most likely do NOT represent the actual performance statistics of your Web server. That's because a single CPU running Pounder II cannot outrun a WebSTAR class server. Other Mac servers like MacHTTP or NetPresenz may be overrun by Pounder II. But to collect accurate stats on WebSTAR, you should consider running multiple copies of Pounder II on separate CPUs.
To run multiple copies and gather accurate results, all copies of Pounder II should begin testing nearly simultaneously. If they can be started within a few seconds of each other and are allowed to run for a significant period of time, the data displays from all copies of Pounder II can simply be added together to get a measure of overall server performance.
Remember that testing over an ethernet is not the same as testing over a dial-up connection. To get accurate results, you should test servers using the actual network connections available to your users.
The log file contains an overabundance of information. With the exception of the SUMMARY information, it will likely be necessary to perform some sort of postprocessing on the log file to extract useful information from it. Sorry, but postprocessing of the log file is beyond the scope of Pounder II. The log file is easily parsed by any number of off the shelf apps. Have fun!
7.0 Legal Stuff
Pounder and Pounder II are products of BIAP Systems, Inc. They aren't affiliated with StarNine Technologies, Inc., Quarterdeck Corp., or Apple Computer, Inc. You shouldn't bother the technical support people at those companies with questions about Pounder II and you shouldn't bother me, either. Pounder II is released "as-is", with no warranties given as to its suitability for a given application. You're on your own! (though plenty of mailing lists and news groups will likely be glad to provide plenty of advice.)
BIAP Systems, Inc. is making Pounder II freely available to all registered users of MacHTTP and WebSTAR. It is free, but it is NOT public domain. BIAP Systems, Inc. holds the copyright and retains all rights to the software and documentation for Pounder II. If you would like to include Pounder II in some sort of electronic distribution, please obtain written permission from the author first. Contact information can be found at http://www.biap.com/
